Online Principal Component Analysis

We consider the online version of the well known Principal Component Analysis (PCA) problem. In standard PCA, the input to the problem is a set of vectors X = [x1, . . . , xn] in Rd×n and a target dimension k < d; the output is a set of vectors Y = [y1, . . . , yn] in Rk×n that minimize minΦ ‖X − ΦY ‖F where Φ is restricted to be an isometry. The global minimum of this quantity, OPTk, is obtainable by offline PCA. In the online setting, the vectors xt are presented to the algorithm one by one. For every presented xt the algorithm must output a vector yt before receiving xt+1. The quality of the result, however, is measured in exactly the same way, ALG = minΦ ‖X−ΦY ‖F. This paper presents the first approximation algorithms for this setting of online PCA. Our algorithms produce yt ∈ R with ` = O(k ·poly(1/ε)) such that ALG ≤ OPTk +ε‖X‖F. ∗Yahoo Labs, New York, NY †Technion Israel Institute of Technology and partially at Yahoo Labs ‡Yahoo Labs, Haifa, Israel §Yahoo Labs, New York, NY